perm filename ARPA.MSG[DOC,TOB]1 blob
sn#154847 filedate 1975-04-16 generic text, type T, neo UTF8
∂15-APR-75 1313 network site ISI
Date: 15 APR 1975 1303-PDT
From: FRASIER at USC-ISI
Subject: DRAFT MINUTES OF THE ARPA IMAGE UNDERSTANDING WORKSHOP
To: TOB at SU-AI, DUDA at SRI-AI, BKPH at MIT-AI,
To: PRATT at USC-ECL, NEWELL at CMU-10A, SELFRIDGE at BBN,
To: STOCKHAM at UTAH-10, TENENBAUM at SRI-AI, PATRICK at MIT-AI,
To: WINTZ at PURDUE
cc: FRASIER
REQUEST:
NEWELL PASS COPY TO REDDY
PATRICK PASS COPY TO EDEN
TOB PASS COPY TO QUAM
PRATT PASS COPY TO ANDREWS
1. ATTACHED IS A DRAFT COPY OF THE MINUTES OF THE 6-7 MARCH 1975
MEETING OF THE ARPA IMAGE UNDERSTANDING WORKSHOP.
2. REQUEST THAT YOU REVIEW. NOTIFY ME (FRASIER AT ISI OR 703-527-7571)
OF COMMENTS OR CHANGES DESIRED BY 18 APRIL, 1975. AT LEAST MAKE SURE
THAT YOU ARE NOT INADVERTENTLY LIBELED.
3. THIS IS A RETRANSMISSION OF THE DRAFT MINUTES. SOME DID NOT
RECIEVE THE ORIGINAL TRANSMISSION OF 4 APRIL, 1975. MY REGRETS TO
THOSE WHO DID.
BEST, CHARLIE
DRAFT
FROM: Chairman, ARPA Image Understanding Workshop
TO: Distribution
SUBJECT: Minutes of the 6-7 March 1975 meeting of the ARPA
Image Understanding Workshop, Promulgation of;
1. The minutes of the 6-7 March meeting of
the ARPA Image Understanding Workshop are hereby
promulgated.
2. In some instances in the attached proceeding,
the contributions of many of the participants
in a discussion have been paraphrased into
one summary statement made by one of the
participants. This stylistic tactic was adopted
to retain the dynamic flavor of the workshop's
discussions without loosing the context in a verbatum
transcriptions. While each participant may not be explicitly
credited in the proceedings for his
contributions,the contributions of all present
were invaluable to the concepts formulated during
the workshop.
3. The next meeting of the workshop will be
ed. note:
The remainder of the letter of promulgation
will be completed in the formal version of the
proceedings.
TABLE OF CONTENTS
ed. note: The table of contents will be completed
in the formal version of the proceedings.
LETTER OF INVITATION
11 February 1975
Dear Sir:
You are invited to participate in an image understanding
workshop at ARPA on 6 March 1975 from 12-5pm and on 7 March 1975
from 9-12 noon.
The purpose of this workshop is to explore possible
future avenues of basic and applied research for the automatic
extraction of information from photographs and other image
data.
Interest in this subject is certainly multi-disciplinary.
Therefore, ARPA has invited representatives of the signal
processing, intelligent systems, and pattern recognition
communities. A list of invited participants is attached.
Since the first days agenda will respond to the views
of the participants, each person is being asked to submit
a two or three page statement on potential research approaches
to image understanding (automatic data extraction). These
papers will be cross-circulated to all participants.
Please submit your statement by 24 February 75 to allow
time for re-mailing prior to the workshop.
Some questions which might be addressed by your brief
statement follow:
1. How complete are the known signal processing
primatives for automatic data extraction.
2. How do the terms semantics and syntax relate to
imagery?
3. Is a linguistic approach to image understanding
credible? If so, is it the best approach?
4. What are the unknowns in the automation of image
understanding?
In the late afternoon of 6 March 75, a casual cocktail
session is scheduled in the Rosslyn area.
The agenda for Friday morning will be self-generated by
the group. This will most likely be an in-depth extension
of the most promising ideas from the preceeding day.
Thank you for your interest. Please submit your
statement by 24 Feb 75 and plan to
attend on 6 and 7 Mar 75. If you plan to attend or
if you have further questions, please contact Maj.
Carlstrom at (202)-694-3049 or ARPANET - Carlstrom
Sincerely,
David L. Carlstrom
Major, USAF
Program Manager
Information Processing
Techniques Office
ATTENDEES TO THE
ARPA IMAGE UNDERSTANDING WORKSHOP
6-7 MARCH 1975
NAME/ADDRESS TELEPHONE ARPANET
MAILBOX
Harry C. Andrews 213-746-5514
USC
Ralph Berstein 301-840-0111
IBM Corporation
Federal Systems Division
Gaithersburg, Md. 20760
T.O. Binford 415-497-4971 TOB at
AI Laboratory SU-AI
Computer Science Department
Stanford University
Stanford, Calif. 94305
Norman Caplan 202-632-5881
National Science Foundation
David L. Carlstom 202-694-3049 CARLSTROM
ARPA/IPTO at ISI
1400 Wilso Blvd.
Arlington, Va. 22209
C.K. Chow 914-945-2264
IBM Corporation
P.O. Box 218
Yorktown Heights, N.Y.
Richard O. Duda 415-326-6200 DUDA at
Stanford Research Institute SRI-AI
Menlo Park, Calif. 94025
Murray Eden 617-253-4185
Massachusetts Institute of Technology
Room 36-621
Cambridge, Mass.
Charlie Frasier 703-527-7571 FRASIER
Science Applications Inc. at ISI
1911 N. Fort Meyer Drive
Arlington, Va. 22209
Berthold K.P. Horn 617-253-6218 BKPH
Massachusetts Institute of Technology at MIT-AI
Artificial Intelligence Laboratory
Cambridge, Mass.
H.K. Huang 202-625-2121
NBRF
Georgetown University Medical School
Washington, D.C. 20007
B.R. Hunt 505-667-4567
Los Alamos Scientific Laboratory
Box 1663
Los Alamos, N.M. 87544
D.H. Janney 505-667-7172
Los Alamos Scientific Laboratory
Box 1663
Los Alamos, N.M. 87544
A.K. Joshi 215-243-8540
University of Pennsylvania
Philadelphia, Pa. 19174
Allen Klinger 213-825-7825
3531 Boelter Hall
University of California, Los Angeles
Los Angeles, Calif. 90024
Russel A. Kirsch 202-921-2337
National Bureau of Standards
Washington, D.C. 20234
Barton Krawetz, LTC USAF 202-695-0822
Office of the Secretary of Defense
Assistant Secretary for Intelligence
Room 3C-200
The Pentagon
Arlington, Va.
J.C.R. Licklider 202-694-4001 LICKLIDER
ARPA/IPTO at USC-ISI
1400 Wilson Blvd.
Arlington, Va. 22209
James B. McFerran 612-456-3347
Sperry Univac - DSD
P.O. Box 3525
St. Paul, Minn. 55165
David L. Milgram 301-454-4527
Picture Processing Labortary
University of Maryland
College Park, Md. 20742
Allen Newell 412-621-6200 NEWELL
Carnegie Mellon University ex. 562 at CMU-IOA
Pittsburg, Pa.
William K. Pratt 213-746-2694 PRATT at
University of Southern California USC-ECL
Los Angeles, Calif. 90007
Lynn H. Quam 415-497-4971
Artifical Intelligence Laboratory
Stanford University
Stanford, Calif.
R. Reddy 412-621-2600
Carnegie Mellon University ex. 149
Pittsburg, Pa.
Azriel Rosenfeld 301-454-4527
Picture Processing Laboratory
University of Maryland
College Park, Md.
Oliver Selfridge 617-491-1850 SELFRIDGE
Bolt, Bernak, Neewman ex. 536 at BBN
50 Moulton St.
Cambridge, Mass. 02138
Thomas G. Stockham 801-581-6885 STOCKHAM
MEB 3160 at UTAH-10
Computer Science Department
University of Utah
Salt Lake City, Utah 84112
J. M. Tenenbaum 415-326-6200 TENENBAUM
Standford Reasearch Institute at SRI-AI
Menlo Park, Calif. 90007
T.J. Wachowski 202-694-5409
AFOSR/NM
Patrick H. Winston 617-253-6218 PATRICK
Massachusetts Institute of Technology at MIT-AI
Artificial Intelligence Laboratory
Cambridge, Mass.
Paul Wintz 317-749-2540 WINTZ at
Purdue (EE) ISI
Lafayette, In. 47904
ARPA IMAGE UNDERSTANDING WORKSHOP
6 MARCH 1975
Papers had been solicited from the attendees to the Workshop in
advance and were published the morning of the meeting. The
attendees to the meeting were given copies of the papers as they
arrived and the first half hour of the meeting was devoted to
reading the papers. The papers that were provided to the
attendees are included as Appendix A to these minutes.
1.0 INTRODUCTION
Major Carlstrom of the ARPA Information Processing Techniques
Office (IPTO) opened the meeting. The attendees were introduced.
(A list of attendees is published on page -- of these minutes.)
1.1 WORKSHOP BACKGROUND
#Major Carlstrom discussed the background which resulted in this
Workshop. ARPA IPTO has had a program entitled "Picture Processing"
for four years which has been involved in the digital coding of
pictures, compressing the bandwidth required to communicate pictures,
and deblurring photographs. At the same time, IPTO's Artificial
Intelligence (AI) activities have contributed to scene analysis.
It now appears that the activities in these areas have progressed
to the point where they could be fruitfully merged in a new program
entitled "Image Understanding"
(IU). It is hoped that this Workshop will identify the problems that
should be addressed by the Image Understanding program, the
unknowns with which the Image Understanding program should be
concerned, and approaches to solving the problem which the Image
Understanding program will address.
1.2 DoD PROBLEMS WHICH THE IMAGE
UNDERSTANDING PROGRAM MIGHT ADDRESS
#Major Carlstrom discussed problems within DoD which might be
addressed by the Image Understanding program.
1.2.2 LOGISTICS
There is a continuing need within DoD to verify logistic documentation.
This is not simply a matter of verifying printed text. Diagrams of
all sorts must also be verified and this presents an opportunity
for Image Understanding.
1.2.3 WEAPON SYSTEMS
Weapon Systems Guidance appears to be an opportunity for Image
Understanding to exploit. Before the widespread use of inertial
navigation, DoD was considering a guidance system called MACE. It
was basicaaly a map matching guidance system. The idea is still
relevant and is receiving renewed interest in the Navy. Automation
of the process is an opportunity for Image Understanding.
Also, target selection is a process which could be automated.
1.2.4 COMMAND, CONTROL, AND COMMUNICATION (C3)
C3 will probably not be a primary concern of Image Understanding.
However, progress in Image Understanding may benefit C3 in such
areas as bandwidth compression. The bandwidth required to
communicate the baudot code for "T38 tank" is orders of magnitude
less than the bandwidth required to communicate sufficient pictorial
information for a human to identify a T38 tank.
1.2.5 OTHER PROBLEMS WITHIN DoD WHICH THE IMAGE UNDERSTANDING
PROGRAM MIGHT ADDRESS
#Major Carlstrom solicited suggestions regarding other problems
within DoD which the Image Understanding program might address.
Suggestions were:
# The dynamic interaction between a commander and
a machine.
# The automatic detection in the data being supplied
to the NCA of the cues which influence the progress
of the SIOP.
# Weather data understanding.
# Cartography.
1.2.6 THE INFLUENCE OF THE PROBLEM DOMAIN ON IMAGE UNDERSTANDING
Major Carlstrom noted that many of the papers submitted by the
attendees discussed the influence of the problems to be
addressed by Image Understanding on the directions that the Image
Understanding program might take. Dr. Rosenfeld's paper was cited
as a good example. In the Image Understanding program it is
desired to devise general solutions that are applicable to large
categories of specific problems.
1.3 THE NATURE OF THE IMAGE
UNDERSTANDING PROGRAM
Dr. Licklider discussed the intended nature of the Image Understanding
program. The program will be conducted over a period of five years.
The objective will be to develop the technology which can be exploited
by the DoD components to solve their specific problems. Thus, the
activities that will be supported in the program will not be the
engineering of specific solutions to specific problems. The
philosophy in the program will be to devlop generalized technology
by driving the research in particular directions. However, at the
end of the five year period the technology developed must be in a
state in which it can be utilized by the DoD components to solve
their specific problems without requiring a significant research
effort to figure out how to apply the technology to the specific
problems. For this reason, the program must result in a demonstration
at the end of the five year period that an important DoD problem
has been solved. The two objectives of developing a generalized
technology and demonstrating the solution to at least one specific
problem are, admittedly, ambivalent. In DoD aaproximately ten
percent of the budget is devoted to R&D. System acquisition accounts
for less than forty percent of the budget and payroll costs account
for the remainder. The competition for the limited monetary resources
available drives research efforts to convincingly demonstrate to their
competition that they are worthwhile. This utility demonstration is
frequently expressed as the useful existing systems which resulted
directly from the research. The rapid assimilation of new concepts
into widely available standard stock items in the computer industry
drives computer research efforts that must demonstrate their
utility to convincing utility demonstations at the end of the research
project that can be directly transitioned to DoD components.
Time sharing and the ARPANET are examples. ARPA was instrumental
in developing time sharing but it has been made available to DoD
through industry. The money spent by DoD through ARPA in developing
time sharing has paid off but it is difficult to defend ARPA
computer research efforts with this example because the DoD
components do not see a direct link. The ARPANET is being
transitioned to DCA directly and the research effort which
developed the technology that the ARPANET demonstrates can be clearly
perceived to have been worthwhile. In the final analysis, ARPA's
utility will be judged by the major DoD problems that are directly
percieved to have been solved by ARPA's efforts.
2.0 THE LINGUISTIC APPROACH TO
IMAGE UNDERSTANDING
#Major Carlstrom stated that he had deliberately emphasized the
lingusitic approach to Image Understanding in his letter of
invitation to challenge the attendees to consider the application
of the Artificial Intelligence technology developed in the Speech
Understanding program to Image Understanding. The papers
received can be divided into the broad categories of those that
agree with the linguistic approach, those that disagree, and those
that agree but feel that linguistiic terminology is inappropriate.
#Major Carlstrom offered his personal view of what is meant by Image
Understanding and why the linguistic approach appears to be
potentially useful. Image Understanding implies applying a
knowledge domain to raw image data in some dynamic fashion - dynamic
in the sense that the knowledge domain is applied in a conditional
fashion rather than a hardwired fashion. The human photo
interpreter applying his knowledge to the raw data in photographs
is an anology. If language is defined as a mapping between
individuals' knowledge domains, then a linguistic approach to an
Image Understanding system which must map to human knowledge
domains is appropriate.
However, there is merit in all three positions taken in the papers
and Major Carlstrom invited a debate on the linguistic approach to
Image Understanding.
2.1 THE APPROPRIATENESS OF THE LINGUISTIC
APPROACH TO IMAGE UNDERSTANDING
#Dr. Eden stated that he had noticed in the papers a global
approach to the use of linguistic concepts to Image Understanding.
This could be a drawback in that it could force Image Understanding
into a preconceived mold. A linguistic approach might be quite
reasonable with conventional symbols. However, natural scenes may
not be expressable as conventional symbols and the linguistic approach
would be unreasonable.
#Dr. Winston commented that if an Image Understanding machine
must communicate to humans, then it must include linguistics.
However, there exist real problems in image processing which must
be addressed in the Image Understanding machine.
#Dr. Newell commented that Image Understanding should concentrate
on the following problem: given that an Image Understanding machine has
a task to perform, what is there in an image that must be
understood in order for the task to be accomplished. In this sense,
the understanding in Image Understanding is whatever it takes to
perform the task.
#Major Carlstrom commented that the name Image Understanding
evolved in the following manner: The IPTO picture processing
program was renamed image processing because it was desired to include
processing of all two dimensional data within the program rather than
restricting it to photograph processing. Now it is desired to
incorporate Artificial Intelligence efforts within the program so the
name has evolved to Image Understanding. The use of the word
understanding should not be allowed to cause the confusion that the
Image Understanding program will concentrate exclusively on the higher
levels as is the case in the Speech Understanding program. There
is still much to be done in signal processing within the Image
Understanding program.
2.2 FIDELITY - A SIGNAL PROCESSING APPROACH
TO IMAGE UNDERSTANDING
#Dr. Andrews stated a signal processing approach to Image Understanding:
1. Establish Image Understanding fidelity criteria.
2. Optimize primitives with respect to the fidelity criteria.
#Dr. Winston commented that this approach might work well once
competing ideas for Image Understanding had been established. The
problem at the moment is that conceptual frameworks for Image
Understanding have not been established.
#Dr. Reddy objected. Fidelity criteria are employed in Image
Understanding now. For example, if a machine can answer the
same questions about an image as human beings, with the same accuracy,
then the machine is an Image Understanding success.
#Dr. Stockham commented that fidelity criteria can be established for
formalized concepts. Within a formalized conceptual framework,
theories can be proved and progress can be measured by testing.
At the moment the Workshop should be considering the interdisciplinary
approach to establishing the formal architecture of Image Understanding.
#Dr. Licklider remarked that tests can be made now. For example,
finding tanks is an objective test of an Image Understanding system
that is more objective than the tests applied in photo enhancement.
#Dr. Stockham replied that performance guarantees which can
be theoretically derived are as valid as performance measured from
sampling testing.
#Dr. Rosenfeld commented that the only way to prove theorems
about tank recognition is with theoretical models of tanks.
#Major Carlstrom commented that both the sampling and theoretical
approaches to performance fidelity may be appropriate.
2.3 REDUCTION OF DIMENSIONALITY AS AN
APPROACH TO IMAGE UNDERSTANDING
#Dr. Wintz stated that Image Understanding should be directed
towards the approach of reducing 10 exp 6 bits of data to simple
statements which convey all the useful information contained in the
raw data. Viewed in this sense, Image Understanding is the process
of reducing the dimensionality of the useful information. At some
point in the reduction process, the linguistic concept of syntax
will become useful.
3.0 TERMINOLOGY
#Major Carlstrom noted that terms such as syntax, primitive, etc.,
had been employed in his letter of invitation, the papers submitted
to the Workshop, and in the preceeding debate. There appeared to be
differing working definitions of these terms being employed. He
asked for a discussion of the definition of the terminology being
employed.
3.1 PRIMITIVES
#Major Carlstrom expressed the opinion that primitives are the
result of a mapping from signal space to symbol space. An edge is a
primitive obtained by a mapping from the signal.
#Dr. Eden opined that there are two classes of primitives:
1. Primitives in the signal space.
2. Primitives in the symbol space.
There is a need for abstract primitives at a very low level. For
example, an edge is not present in a picture. Edges are created through
processing by the eye. Also, primitives may be task oriented.
Different tasks may require different primitives.
3.2 SYNTAX, SEMANTICS, PRAGMATICS
#Major Carlstrom expressed the opinion that syntax is the structural
element of an image.
#Dr. Licklider remarked that in the unity of science movement of
the 1930s, syntax, semantics, and pragmatics had widely accepted
formal definitions:
Syntax - the structural relationship between signs.
Semantics - the mapping of data into signs.
Pragmatics - the use of signs.
3.3 THE UTILITY OF THE TERMINOLOGY
#Dr. Horn expressed the opinion that the terminology cannot be
used absolutely. Syntax is a name loosely applied to a bag of tricks
which may help with Image Understanding. However, what Image
Understanding interests should be, in a sense similar to syntactics,
is in how images are formed and what are the constraints structuring
them.
#Dr. Newell commented that this approach is similar to analysis by
synthesis used in Speech Understanding.
#Dr. Tenenbaum remarked that using the meaning of objects to define
their attributes is a semantic approach. In this approach,
objects would be tentatively identified at an early stage in image
processing. For example, in region growing processing, objects would
be labelled at the earliest point at which there is a good probability
of correctly labelling the object. Region growing would then
continue in areas of the image not previously labelled until further
object identification could be made.
#Dr. Newell commented that in Speech Understanding, people use
semantics and syntax in a nominal way. When pressed, they retreat
from this terminology and express their ideas in terms of sources of
knowledge. There exist a spectrum of concepts which cannot be
adequately expressed with the syntax/semantics terminology.
#Dr. Joshi commented that the Workshop should be interested in the
interaction of knowledge sources and should not encumber itself with a
restricted terminology.
4.0 THE COMPLETNESS OF KNOWN SIGNAL
PROCESSING PRIMITIVES FOR AUTOMATIC
DATA EXTRACTION
#Major Carlstrom stated that in response to his question on the
completness of primitives in the letter of invitation, some
papers had replied that known signal processing primitives are
adequate and some had replied that they are inadequate. He
entertained a debate on the subject and proposed that there are two
primitives:
1. Edge detection.
2. Region growing.
#Dr. Binford commented that color, texture, shape, and scale are
primitives.
#Dr. Kirsch commented that a useful set of primitives could only
be identified by considering classes of images at several levels of
complexity. Useful primitives would allow mapping from one level
to the next.
#Major Carlstrom expressed his philosophy in attempting to identify
primitives. If the Artificial Intelligence community could identify
sufficient sets of primitives needed in Image Understanding systems,
then the signal processing community could set about extracting those
primitives from raw data.
#Dr. Pratt expressed the opinion that if analytical techniques were
to have impact in Image Understanding, then the level of primitive
that should be used is at the edge, texture, and color level.
#Dr. Reddy commented that a program was being conducted at Carnegie-
Mellon University to attempt to identify the primitives that people
are interested in. The technique employed is to allow subjects to ask
questions about a picture they cannot see. The subjects are supposed
to identify the picture from the answers to their questions. People
do not ask questions about texture and from this it appears that
texture is not a primitive that is useful to people. The color of
various regions of the picture is a question frequently asked. Blue
in the upper portion of the picture appears to be a discriminator for
sky. Dr. Reddy said that he would make the list of primitives obtained
by this program available to the attendees of the Workshop.
#Dr. Rosenfeld suggested that a list of useful primitives might be
constructed by interviewing artists who compose pictures.
#Dr. Licklider remarked that Cezanne considered the cube, trapezoid,
and cylinder to be the useful primitives.
4.1 A BIOLOGICAL APPROACH TO
IMAGE UNDERSTANDING
#Mr. Frasier commented that the approach to Image Understanding
expressed by the Workshop could be characterized as platonic.
There appeared to be an assumption that there is a reality of things
as they are which underlies images and the Workshop's approach is
to discern this reality from the shaddows on the cave wall. He
proposed a biological approach. The machine already exists which
can perform all the functions desired of Image Understanding machines.
It is the human nervous system. The Image Understanding program should
concern itself with copying the human system.
#Dr. Winston objected to this concept on the grounds that so little is
known about the human nervous system that it is impossible to copy.
#Mr. Frasier replied that the tools which will allow copying portions
of the nervous system are rapidly evolving. Dr. Stephen
Grossberg of MIT has developed a theory for the operation of the
nervous system that can theoretically model some of its functions.
Dr. Stanley Crain of the Albert Einstein College of Medicine can
maintain neural tissue segments in vitro and conduct controlled
tests to determine the segments' functions. Efforts at mapping
neural signal flow are continuing. Applying Grossberg's theory to
the signal flow data and conducting critical tests on tissue
segments may be a very feasible approach to understanding the nervous
system in a manner that will allow it to be copied in machines.
The difficulty with this approach is not technical. The
difficulty is that the signal flow mappers and the tissue segment
testers are funded by NIH. They are motivated to use their
knowledge to cure disease rather than to build Image Understanding
machines. ARPA funding in this community might have some large
payoffs.
#Dr. Newell commented that even if the state of knowledge was as
advanced as Mr. Frasier implied, there would be difficulty in obtaining
the processing capacity required to model the nervous system.
#Mr. Frasier remarked that modelling the optic tract might be very
difficult, from a capacity standpoint, on a sequential computer.
The optic tract consists of many parallel channels which are processed
in parallel simultaneously. Perhaps extension of parallel
processor architecture should be considered to provide machines which
are better suited to modelling the nervous system.
#Dr. Eden commented that, in line with his previous statements,
there might be different sets of primitives: those that are useful for
machines and those that are useful for neurophysiological systems.
#Dr. Pratt stated that there have been some useful results to Image
Understanding from efforts which modelled biological systems.
Photograph enhancement and deblurring projects
have found that a process in which the raw data in illuminance
space is mapped via a filter which models a human eye with a laterally
inhibiting retina into geodesic or chromodesic space is useful.
These efforts have been conducted at the University of Utah.
The workshop was adjourned for a coffee break.
5.0 POTENTIAL PROBLEM AREAS OF
INTEREST TO IMAGE UNDERSTANDING
The Workshop reconvened.
#Major Carlstrom stated that the Agenda Items "Issues in Image
Semantics" and "Issued in Image Syntactics" had been discussed in
the preceeding topics. During the break several of the attendees had
urged a discussion of the problem areas that the Image Understanding
program should address. He had decided, therefore, to replace the
scheduled Agenda Topics with a discussion of potential problem
areas that might be addressed by the Image Understanding program.
He presented a list of potential problem areas which was as follows:
Potential Problem Areas of Interest to Image Understanding
(Spanning Set of Tasks)
1. Photo Intelligence
2. Geophysical (ERTS, LANDSAT)
3. Cartography
4. Meteorology
5. Remotely Piloted Vehicles (Robotics, Guidance)
6. Surveillance.
5.1 REMOTELY PILOTED VEHICLES
An objection was raised to including remotely piloted vehicles (RPV)
but it was concluded that RPVs are an attractive problem since they can
directly evidence Image Understanding.
5.2 PHOTO INTELLIGENCE
#Dr. Tenenbaum discussed the automation of Photo Intelligence functions
with an illustration: Determining landmarks in an ERTS photograph of the
San Francisco Bay area. From general knowledge of the Bay area obtained
from travelling in the area and using road maps, he knew the approximate
outline of the peninsula. In the photograph this outline showed up
because the peninsula appears very linear, whereas the water surrounding
it was less organized. With respect to the peninsula's outline, he
knew approximately where to look for the airports in the area. These
showed up as distinct patterns. Having located the airports and using
his knowledge from travelling in the area, he was able to locate a
pattern which conformed to his expectation of what Candlestick Park
should look like. From his knowledge gained by travelling in the
Candlestick Park area and expectations about what large parking lots
look like, he was able to identify a shopping center near Candlestick
Park. Using this shopping center's pattern, he was able to locate
additional shopping centers.
This example represents the use of partial knowledge to identify the
contents of an image. Automatically this process would be a
useful approach to automating Photo Intelligence.
An objection was raised that Dr. Tenenbaum was finding in the photograph
what he knew to be there. Photographic interpreters are interested
in finding new objects in photographs.
#Dr. Bernstein commented that, in his experience with ERTS photographic
interpreters, they followed the same approach as Dr. Tenenbaum.
The photo interpreters orient themselves with respect to familiar
features and then proceed to identify new objects. He recommended
that Dr. Tenenbaum's approach be applied to each of the potential
problem areas to identify the subtasks which needed automating to
accomplish the overall objectives.
5.3 THE PROCESSING CAPACITY REQUIRED
FOR IMAGE UNDERSTANDING
#Dr. Wintz commented that he had estimated the processing capacity
required to automate Dr. Tenenbaum's algorithm and had found the
demands too great for a sequential machine. For example, airports
can appear on a photograph at any location and in any orientation.
Each orientation at each location must be checked on a photograph.
#Dr. Quam commented that the computational complexity could be
reduced by applying knowledge about the photograph in a top down
fashion. For example, the altitude from which the photograph was
taken would be known as well as the geographic orientation of the
photograph and the approximate location at which the photograph was
taken. This knowledge would limit the orientation and projection
size that distinguishing landmarks could take in the photograph.
Since the entire area of the photograph would not be of interest,
identification of several distinguishing landmarks would bound the
areas of interest for more detailed analysis and thereby limit the
computational requirements.
#Dr. Stockham asked how many pictures a day must be processed by
photographic interpreters. He proposed, for the sake of eliciting a
discussion, that no matter how simple a task in photographic
interpretation was to be automated, there did not exist sufficient
computer power to perform the task in real-time.
5.4 A SPECIAL PURPOSE PROCESSOR FOR
IMAGERY INFORMATION EXTRACTION
#Dr. Swanland described the efforts at CDC with a special purpose
processor for automated imagery information extraction in reply to
Dr. Stockham's assertion.
5.4.1 INTRODUCTION
Three applications of imagery information extraction have been
implemented on a special purpose processor array. These
applications are synthetic aperture radar change detection, infra-red
mapper object detection and orthographic data generation. The
change detection system includes major subsystems of coordinate
transformation, scene registration, tonal normalization and change
analysis. These applications utilize high throughput processing to
extract a limited set of information categories.
It now seems appropriate to initiate a second generation processor
array. To do this should include consideration of a larger set of
applications. Then the processor will become more general purpose.
The question then is appropriately, how general can the machine
become without sacrificing the cost and efficiency of the special
purpose processor. Further, it is desirable to develop a
convenient software interface to the designer/researcher and to
incorporate a wide variety of information extraction techniques.
5.4.2 CURRENT STATUS
One current array processor has an equivalent throughput of 3.2 x 10
to the 8th operations per second. For example it could perform 32 x
10 to the 8th additions per second while simultaneously performing
1.6 x 10 to the 8th multiplications per second. In an actual system,
there is some mix of instructions. To get some representation number,
the system is rated in terms of the maximum number of additions it
could perform.
The applications of this array include a significant number of
imagery processing functions. These include the functions listed
in Table 5-1. In all cases, the equipment is intended to keep up
with the sensing rate.
Some observations based on the experience to date is summarized in
Table 5-2. The architecture of the array processor is tailored
to those functions which are repetative across all or large portions of
the image. Of the functions considered, the input window is within
16,384 pixels. The window is moved progressively across the image.
Within the window, the operations are repetitive on a sequence of
pixels. Thus a microcoded algorithm can operate repetitively on a
large sequence of data samples. To date the majority of the
operations have only required 16 bit arithmetic capability.
TABLE 5-1. CURRENT TECHNOLOGY
BUILT: 320 MOPS MICRO-PROGRAMMABLE PROCESSOR ARRAY
APPLICATIONS:
# RADAR CHANGE DETECTION
# SCENE REGISTRATION
# GREY SCALE NORMALIZATION
# ORTHOIMAGE GENERATION
# OBJECT DETECTION/CLASSIFICATION
# AREA CLASSIFICATION (MULTISPECTRAL)
TABLE 5-2. SOME OBSERVATIONS
1. ARCHITECTURE FITS APPLICATIONS
# 128 x 128 to 1 x 16,384 INPUT WINDOW
# REPETITIVE OPERATIONS PER PIXEL AND HIERARCHICAL
# STAGES --> PARALLEL/PIPELINE
# 10 - 10 to the 3rd OPNS/PIXEL --> MODERATELY GP
# THROUGHPUT --> HIGH SPEED I/O, TRANSFER
# ASYNCHRONOUS --> ARRAY FLEXIBILITY
2. SOFTWARE COSTS HIGH
# 25X FORTRAN FOR 50% EFFICIENCY (1 HR/INST)
# 100X FORTRAN FOR 90% EFFICIENCY (4 HR/INST)
# 40% ON LINE SYSTEM COST (50% INCLUDING ALG. DEVELOPMENT)
3. APPLICATION - DIRECTED DESIGN
# 1/5 - 1/10 DEVELOPMENT COST
# 1/10 - 1/30 BUILD COST
# LARGE PROPORTION IMAGE PROCESSING APPLICATIONS
The large data throughput requirements are too great for a single
processor. Thus it was necessary to modularize the processing into
an array. The array is arbitrarily parallel, pipeline or a mixture
of the two. Thus if the processing is complex, modules can be
pipelined to keep up with the pixel rate. If the processing is
simpler, then throughput rate is increased by parallel processors
operating on separate data streams.
A useful criteria for processing complexity is operations per
pixel. By reducing all operations to the equivalent addition
operation, a single variable is achieved. For example, in the
example processor, a multiply takes two instruction cycles and is
equivalent to an add which is one instruction cycle. The functions
being considered require 10-10 to the 3rd operations per pixel.
The throughput processing load is then the product of the input rate
times the operations per pixel.
In summary, processor modules are made efficient for repetitive
operations on pixels. Thus, high speed input/output and transfer
between arithmetic units is important. Also the processors in the
array are asynchronous to minimize the difficulties in timing of
processor arrays.
The processor array was designed for one application. To minimize
development costs, only minimal software aids were developed. The
penalty for machine language coding is well recognized. In this
case we have found that it takes 1 hour per instruction to achieve
50% processor efficiency in the system, 90% efficiency increases the
time to 4 hours per instruction. These are highly qualified
programmers who estimate that equivalent FORTRAN coding would be
approximately 2 minutes per instruction. These estimates are for
coding after the algorithms have been defined. The cost of the
microcoding is about 40% of the total system cost. If one included
the cost of the algorithm development, the software cost is about
50% of the total development cost.
It is interesting to note that the development and build cost is
greatly reduced over GP machines. This is a difficult comparison
to make since the development should be at the same time and using the
same technology. The total cost of development and one prototype
system is about $4M. This can then be compared against other large
general purpose systems aimed at high data throughput. Candidates
for comparison are the STARAN, PEPE, ILLIAC IV and STAR-100 systems.
It is also interesting that the special purpose processor array is
able to handle a large number of useful image processing functions.
That is, those which involve some specified processing over the entire
image such as screening for objects, area classification with texture
and multispectral features, elevation measurements from stereo pairs
and registration of two images of the same ground area.
5.4.3 SOME NEEDS
Our experience with designing information extraction systems has
led to some conclusions on current needs. One of these needs is
for an efficient design facility. We have selected the following
as desired characteristics for this facility. These are summarized
in Table 5-3.
1. A high speed interactive imagery/graphics display is needed
for experimental evaluation to determine which features are
effective as primitives, for setting thresholds, for labelling
objects or areas such as in training a classifier and for
adding context from other sources. A reasonable goal is to
access a new image within 1 second.
2. A moderately high speed processor is needed to evaluate
processing algorithms on a statistically significant data
base. For example, 100 images require 3 x 10 to the 9th
operations. A 10 to the 7th operations/second processor
would take about one hour to perform this evaluation.
From a system design point of view, the processor array
would permit timing studies. This is important to determine
if there are timing inconsistencies within a processor as
well as between processors.
TABLE 5-3. EXPERIMENTAL DESIGN FACILITY NEEDS
A. HIGH SPEED (1 SECOND) INTERACTIVE IMAGERY/GRAPHICS DISPLAY
# SELECT FEATURES
# SET PARAMETERS (E.G., THRESHOLDS)
# LABEL OBJECTS
# ADD CONTEXT
B. 10 - 30 MOPS PROCESSOR
# STATISTICS ON FEATURES/CLASSES, E.G.,
100 IMAGES --> 30 BOPS --> 1 HOUR
# SYSTEM TIMING SIMULATION
C. FAST TURNAROUND
# ON LINE FILM READER
# HIGH SPEED TAPE/DISK/RAM SYSTEM
# "QUICK LOOK" IMAGERY COPY, DISPLAY
3. A major stumbling block in most laboratories is poor turn
around times. A self contained system with imagery input,
high speed processing, on line display and convenient software
can greatly reduce the turn around time. By also keeping the
cost down it is easier to justify "hands off" control of the
entire process. The availability of such a system could greatly
enhance imagery information extraction system design.
Based on the present experience, the next generation processor
array will be improved in several ways. It appears that two
directions are appropriate. One is to push for even higher
throughput rates. A 10 to the 9th operations/second processor array
is feasible with improved componentry and architecture. Also a low
cost array with, say, a 3x10 to the 7th operations/second may be
appropriate for many applications. (This is summaarized in Table 5-4.)
In both cases, flexibility and programming simplicity is needed.
Flexibility is achieved within a processing module by allowing a range
of instruction mixes such as the proportion of arithmetic or logic
submodules. Also the amount of memory or I/O could be specified.
A microassembler within the module would permit the host computer
to transfer macros, rather than detailed microcode instructions.
The array of modules is to be selectable into arbitrary parallel/
pipeline configurations. The reconfiguration of modules should be
under program control. Other potential features such as virtual
memory within the processor may prove to be significant.
TABLE 5-4. NEXT GENERATION
A. HIGH THROUGHPUT SYSTEM - 1 BOP
B. MODERATE THROUGHPUT SYSTEM - 3 x 10 to the 7th MOP
C. MORE GENERAL PURPOSE MODULE
# SPEED VERSUS COST SELECTION
# SELECTABLE INSTRUCTION MIX
# SELECTABLE MEMORY, I/O
# MICROASSEMBLER
D. MORE GENERAL PURPOSE ARRAY
# ARBITRARY ARRAY CONFIGURATION
# DYNAMIC RECONFIGURATION
# VIRTUAL MEMORY
Let us assume that continued development of a more general purpose
array is the correct route. Then a second generation array might be
developed as shown in Figure 5-1. Additional applications are
already providing an impetus for more flexibility. Research in
information extraction algorithms would affect the architecture of
the machine. It would also provide a more general set of application
software for information extraction applications. As part of the
information extraction research, it should be possible to develop
a language suited to this particular community.
The above research and development should be coordinated with the
array architecture and component development. In this way, we can
achieve the dual capability of a general purpose and low cost
imagery processing system. The effect on progress in this field
could be dramatic.
ED NOTE: Figure 5-1 will be included in the formal version
of the minutes.
5.5 IMAGERY DATA BASES
#Dr. Pratt asked if the data processed by CDC could be made available to
the workshop attendees as a data base to see if their algorithms would
work better in some instances.
#Dr. Swanland replied that CDC had not processed much data as yet.
#Dr. Kirsch asked if there were standardized imagery data bases which
Image Understanding research efforts could use as a standard of
comparison.
#Dr. McFerran commented that the pattern analysis committee of the
IEEE has attempted to assemble standardized data bases. Currently
they have data bases in:
# Fingerprints
# Handprints
# OCR
# Analogue Speech.
They have just received the Postal Service's character data bases which
contain 32,000 characters.
#Dr. Newell commented that the Speech Understanding program had
attempted to maintain a standardized speech data base. The effort
was not successful because each contributing researcher tended to
use his own labelling scheme that made the most sense for his particular
research effort. He ventured that attempts to maintain a standardized
imagery data base would encounter the same problems.
6.0 UNKNOWNS IN IMAGE UNDERSTANDING
#Major Carlstrom requested the Workshop's opinions of the unknowns
in Image Understanding.
#Dr. Stockham suggested that imagery models might be an unknown which
could limit the progress that can be made in Image Understanding.
There have been human speech models for thirty years and these
models were available to the Speech Understanding program when it
began. These do not exist in image models.
#Dr. Rosenfeld commented that the labels applied to images exist in the
eye of the beholder. Work is needed in objectively modelling images.
#Dr. Lickliker commented that perhaps the imaged object should be
modelled.
7.0 CONCLUSION OF THE 6 MARCH 1975
WORKSHOP SESSION
Topics which should be included on the agenda for the next day were
discussed. The topics selected, together with their rationale are
presented in the next section.
The 6 March 1975 Session was adjourned.
7 MARCH 1975
8.0 AGENDA FOR 7 MARCH 1975
#Major Carlstrom opened the meeting by introducing the agenda and
discussing the agenda topics. Agenda topics were:
# Specific examples in Image Understanding to focus
the Workshop's effort.
# Further discussion of unknowns in Image Understanding.
# A future meeting.
A criticism of the previous day's proceedings had been that the
discussions were too general. Identifying specific examples in
Image Understanding and limiting the discussion to these examples
would be more fruitful.
The discussion of unknowns in Image Understanding in the preceding
day's proceedings had come at the end of the day and was not
very productive.
The purpose of this meeting, the initial Image Understanding
Workshop meeting, was to familiarize the participants with the
Image Understanding program and to exchange ideas on the subject.
Time is needed to reflect on the discussions of this meeting before
productive concepts for the Image Understanding program can evolve.
It is proposed to have another Workshop meeting in two or three
months. The specific Image Understanding examples would be
studied by the participants and they would discuss their approaches
to the examples at the next meeting.
9.0 SPECIFIC EXAMPLES IN
IMAGE UNDERSTANDING
(The potential problem areas in Image Understanding presented in
Section 5.0 of these minutes are frequently referred to in the
following discussions. The problem areas were on the blackboard in the
conference room during the discussions.)
9.1 DISCUSSION OF THE CRITERIA FOR
SELECTING SPECIFIC EXAMPLES
#Dr. Winston stated that the Artificial Intelligence community could
make greater contributions to the Image Understanding program if the
three dimensional problem areas were selected as specific examples
rather than two dimensional problem areas.
#Dr. Selfridge commented that the specific examples should be selected
on the basis of their potential intellectual fertility. That is,
the problem areas most likely to produce general results of wide
applicability should be favored. In his opinion, three dimensional
imagery appears to be a richer problem than two dimensional imagery
due to its greater complexity. In posing intellectual problems,
the greater the complexity, the better.
#Dr. Tenenbaum stated that the Photo Intelligence, geophysical, and
cartography problem areas were areas in which humans use inference
from knowledge bases. Current Artificial Intelligence work in
automating interpretive processes could be applied in these areas.
#Dr. Kirsch suggested that a criteria for selecting Image Understanding
examples should be whether the human experts performing the tasks had
developed a language so that they can talk about their task.
#Major Carlstrom commented that photographic interpreters have
developed such a language. However, photographic interpretation might
be a difficult example because of security considerations even though
it would be an example of great interest to the Department of Defense.
#Dr. Licklider commented that in designing the Image Understanding
program, a consideration for selecting Image Understanding examples
should be how close one is to a solution in the problem area. If the
problem area is ripe for a solution and an accomplishable goal can be
established, then the program should be designed to accomplish the
goal. Otherwise, the nature of the program will be quite different.
#Dr. Horn commented that the nature of tasks in photographic
interpretation are similar to finding missile silos or tanks.
These tasks are very different from finding shopping centers in San
Francisco because they are not context rich. Tanks or silos could
appear anywhere in a ground scene without a contextual relationship
to the features of the ground scene.
#Dr. Licklider commented that photographic interpreters do not deal
exclusively with a single picture. They deal with a time history of
pictures of the same ground scene. Context is built up through the
history.
#Major Carlstrom stated that this is true. Photographic interpreters
infer the presence of an unseen object through the history of
construction and transportation events which have occurred on a
particular ground scene.
#Dr. Andrews commented that this was net intelligence assessment,
not photographic interpretation.
#Dr. Licklider replied that the Artificial Intelligence community
could make a great contribution to the Department of Defense by
putting the intelligence assessment process into the analysis of
raw data.
#Dr. Stockham commented that there existed many unsolved problems
which do not require the application of Artificial Intelligence to
raw data.
#Dr. Eden commented that an analogy to X-ray interpretation, with
which he has had experience, would illuminate this issue. The
purpose for looking at X-rays makes a great difference to the analysis
techniques applied. Techniques to enhance X-rays to help identify
tuberculosis are disasterous if applied when the purpose of looking
at the X-ray is general diagnosis.
Dr. Chow commented that if the purpose of looking at the X-ray is
to identify a specific feature, then automated systems can do it more
reliably than radiologists. However, automated systems do not do
as well at general diagnosis as radiologists with ten years of
formal training.
#Major Carlstrom commented that this issue is a recurrent problem in
the Department of Defense. Overengineered specific problem solving
machines are abundant. He expressed the hope that the application of
Artificial Intelligence in Image Understanding could generalize these
machines to have wider applicability.
#Dr. Rosenfeld commented that Artificial Intelligence techniques have
been applied to the general diagnosis problem at the University of
Missouri. Regina Bouchi has used very simple Artificial Intelligence
teccniques to identify bridges in ERTs photographs. If the Artificial
Intelligence community were turned on to imagery problems in a
significant way, much more powerful tools for automating Image
Understanding tasks might result.
9.2 SOME IMAGE UNDERSTANDING SYSTEMS
#Dr. Wintz proposed a division of Image Understanding into problems
which ARPA can attack. He presented Figure 9-1 to the Workshop.
The "pre-processor," "segmentation," and "classify some regions"
boxes would reduce the dimensionality of the problem. Artificial
Intelligence would be applied in the "structural analysis" box
with the feedback shown to earlier stages of the process. In this
process, specific problems are:
# Image Segmentation
# Edge Extraction
# Texture Analysis
# Shape Analysis
# Syntactic Analysis.
REAL PRE CLASSIFY STRUCTURE
WORLD->[SENSOR]->[PROCESSOR]->[SEGMENTATION]->[ SOME ]->[ ANALYSIS ]
| ↑ | ↑ REGIONS | |
| | | | | ↑ | |
<---------- ------- ------- -------- |
DESCRIPTION OF REAL WORLD <---------------------
Figure 9-1
#Dr. Selfridge remarked that he had seen similar diagrams in
Speech Understanding. In his opinion, he does not believe that we are
within striking distance of solutions to these problems.
#Dr. Reddy stated that Dr. Wintz's approach was an honest bottom up
approach to the problem. He provided the attached briefing based
on his scene analysis activities. His approach is not hierarchical
and is best described by reference to the paper by Newell and
Reddy in Appendix A.
Dr. Reddy's briefing charts are as follows:
CHART 1
TYPICAL PROTOCOLS
SUB: do the rectangles, and the two greyish blue regions below
blueband make up more than 70% of the area below blueband?
EXP: looks like between 70 and 80%.
SUB: describe one of the remaining regions down there.
EXP: the largest one is a green region bordered with darker
green and grey linear regions, it is irregular in shape and has
patches in it.
SUB: give me location of this green region and approximate size
(vert and horiz bounds).
EXP: it borders left edge of the picture starting from point
1/4 inch above the lower-left corner and extending to a point
1-1/4 above the lower-left corner. It extends to right almost
with constant width (1 inch) for about 1 inch then starts to
diminish in width until it tapers into a wedge like shape the
tip of which is about 4 inches from left edge.
SUB: is this green region flat in 3-D?
EXP: yes.
SUB: is it textured?
EXP: yes.
SUB: thin brown vertical segments in the texture?
EXP: no. Vertical in 2-D?
SUB: yes in 2-D.
EXP: no.
SUB:;CONJECTURE: ok, not trees but probably just grass. I don't
feel any need to continue at this point in the lower region of the
scene. It's a city scape. Back to get
CHART 2
A SAMPLE OF PICTURE PROCESSING OPERATORS:
EQUALITY:
equal?
same?
different than X
X repeated
QUANTITY:
how many?
good deal of
lot of
predominant
more or less
UNIFORMITY:
uniform?
same throughout?
irregular
straight
homogenious
bounded by lines
significantly different
continuity
consistent
ORIENTATION:
with respect to X?
horizontal
vertical
T-junction
parallel
angles
TEXTURE:
what texture?
COLOR-CONTRAST:
what colors?
pronounced
white
blue
green
brown
grey
silver
pink-pale pink
red
yellow
clear
LOCATION:
where?
how far?
left-right
top-bottom
upper-left
below-above X
close to X
where with respect to
contained in
part of
adjacent to
separator of
joined to
non-overlaping
SIZE:
how large?
largest
larger
percent area of
across the picture
length
how far?
SHAPE:
what shape?
circular
solid
length/height ratio
convex
indeterminate
rectangular
trapezoidal
elipsoidal
triangular
CHART 3
ENVIRONMENT TO SEGMENT (GENERATIVE)
City-scape contains clusters of vertically oriented rectangular shapes.
SEGMENT TO ENVIRONMENT (ANALYTIC)
Vertically oriented rectangular clusters are ccntained in city-scapes.
ENVIRONMENT TO OBJECT (GENERATIVE)
Outdoor scene has sky.
Indoor scene has walls.
ENVIRONMENT TO STRUCTURE (GENERATIVE)
Outdoor scenes have lots of greenery.
Outdoor scenes have buildings.
STRUCTURE TO OBJECT (GENERATIVE)
Tires are located around hubcaps.
Sky has clouds.
STRUCTURE TO SEGMENT (GENERATIVE)
(**) Multiple-lane roads are divided by boundaries (lines).
(*) Car wheels are closer to the ground than the car body.
Buildings have lots of vertical lines.
(*) Rectilinear objects occluding each other have straight line
poligonal surfaces with adjacent edges.
(*) Sky and landscape form curvilinear edges - as opposed to
manmade objects.
(*) Bushes are closed to bottom of picture.
(*) Shadows of objects touch bases of objects they belong to.
(*) Manmade shapes are repeated (in a scene).
SEGMENT TO STRUCTURE (ANALYTIC)
(**) A boundary, if appropriately located with respect to a road,
may indicate that the road is multiple-lane.
Lots of vertical lines may belong to buildings.
STRUCTURE TO FEATURE (GENERATIVE)
Greenery is green.
Human's complexion is pink or pale pink.
Sky and distant objects do not form contrast edges. (Given they
have the same color.)
An object occluding another is closer to the observer.
FEATURE TO STRUCTURE (ANALYTIC)
Green is greenery.
Pink or pale pink is human complexion.
Contrast edges are not formed by sky and distant objects.
The object closer to the observer is the object occluding the
more distant one.
OBJECT TO REGION (GENERATIVE)
Buildings have rectangular surfaces.
Rectilinear objects are made up of straight line quadrilaterals
which share sides.
Rectilinear objects' un-occluded surfaces are trapezoidal in
2-D representation.
REGION TO OBJECT (ANALYTIC)
Rectangular surfaces belong to buildings.
Straight line quadrilaterals which share sides are parts of
rectilinear objects.
Trapezoidal surfaces are rectilinear objects' un-occluded
surfaces.
OBJECT TO STRUCTURE (GENERATIVE)
Roads are closer to bottom of a picture.
Eyes are contained in humans.
SEGMENT TO OBJECT ((ANALYTIC)
A blue segment at top of picture is sky.
A large white surface is walls.
A checkerboard texture is probably a building facade.
Trapezoidal surfaces are the faces of rectilinear objects
in perspective.
Edges (lines) vanishing to a common point are edges of
objects in perspective.
"T-junctions" at corners of rectangular surfaces are corners of
rectilinear objects.
An object consisting of elongated brownish vertical rectangle,
to the top of which is attached a more or less convex green mass
of indeterminate shape is a tree.
OBJECT TO SEGMENT (GENERATIVE)
Sky is at top of picture and blue.
Walls occupy a largrr picture area and are white.
(*) Telephone wires run horizontal.
Roads have edges (lines).
(*) Roads have cracks.
Building facades have checkerboard texture.
(*) Water body is a flat surface. (Given it is greyish blue and the
environment is downtown.)
Rectilinear objects have trapezoidal faces in perspective.
Objects in perspective have edges (lines) "vanishing" to a common point.
Corners or rectilinear objects in perspective are "T-junctions."
A tree is an "object consisting of elongated brownish vertical
rectangle, to the top of which is attached a more or less convex green
mass of indeterminate shape."
OBJECT TO FEATURE (GENERATIVE)
Soil is brown.
(*) Roads have uniform color and texture.
(*) Tires have a defined range of colors.
Clouds are white.
Surfaces sloping away (up) from the observer (hills, etc.)
are closer at bottom to the observer than at top.
(*) Grass is flat in 3-D.
(*) Grass is textured.
FEATURE TO OBJECT (ANALYTIC)
If a region is brown then it is possibly soil.
If a region is white then it is possibly clouds.
A surface closer to the observer at the bottom than at the top is an
upward and away sloping surface (hill, etc.)
FEATURE TO OBJECT (GENERATIVE)
Green is greenery.
SEGMENT TO REGION (GENERATIVE)
L-shaped (or its rotations) surfaces belong to occluded
quadrilaterals.
SEGMENT TO OBJECT (GENERATIVE)
A pair of circular objects side by side are eyes (or eye glasses).
REGION TO SEGMENT (GENERATIVE)
Two segments of an occluded region have the same color.
#Dr. Klingle commented that the Image Understanding effort should
be developing better primitives to reduce analysis tasks such as
that briefed by Dr. Swanland the day before. He presented
Figure 9-2 to the working group as a refinement of Figure 9-1
and the idea presented by Dr. Wintz.
ED NOTE: Figure 9-2 has too many interconnections to be
faithfully reproduced on a computer terminal. It will be
provided in the formal version of the minutes.
#Dr. Licklider questioned the output of Figure 9-2.
#Dr. Klingle stated that perhaps global analysis and structure
location were the output.
#Dr. Stockham questioned the feasibility of implementing such a system
in a computer that would keep up with the work provided to it in real
time.
#Dr. Bernstein briefed an approach being studied at IBM. The
task is to find copper ore bearing regions from an ERTs photograph.
The approach is diagrammed in Figure 9-3.
The input consists of 32M bytes of data. Known copper deposits were
used to develop the features used by the process to identify ore
bearing regions. The machine identified 32 possible ore bearing
regions.
The man rejected ten of these based on his knowledge.
As with Dr. Swanland's system, a language is needed to reduce the time
required to program the machine.
#Dr. Reddy suggested that if a man is applying his knowledge to
reject identified ore regions, it might save processing time and
complexity if the man called out unpromising regions before the machine
began processing.
{MACHINE} {MACHINE} {MAN}
INFO IMAGE
IMAGE->[PREPROCESSING]-->[EXTRACTION]------>[UNDERSTANDING]--
DATA |
* FORMATTING * MULTISPECTRAL * INFO |
* CORRECTION CLASSIFICATION DISPLAY |
* ENHANCEMENT * FAULT * INFO |
* LOCATION LOCATION INTERP |
* FEATURE |
INVENT |
* PHENOM |
ISOLATION |
|
------------------------------
|
V
PHENOMENA INFO
{MACHINE AIDED MAN} [MODELING]----->[APPLICATION]
* PREDICTION
Figure 9-3
9.3 REMOTELY PILOTED VEHICLES (RPVs)
#Dr. Pratt remarked that from the foregoing discussion, it appeared that
photographic interpretation should be one of the specific examples in
Image Understanding. He asked Dr. Licklider if RPVs were also of
interest. He stated that the Navy and the Air Force are greatly
interested in RPVs.
#Dr. Licklider replied that RPVs are of interest. The ARPA Tactical
Technology Office (TTO) is at the moment funding image processing
efforts for application in RPVs.
9.4 CARTOGRAPHY
#Major Carlstrom remarked that fatigue plays an important part in the
productivity of photographic interpretation. One objective in
applying Image Understanding to photographic interpretation might be
to automate the boring and redundant tasks so that the humans'
interest could be maintained.
#Dr. Rosenfeld commented that this was true and that the Image
Understanding program should concentrate on the boring, uninteresting
tasks. Some of the tasks in cartography are of this nature.
Some of the real problems in cartography are inputting map data into
computers and updating maps with new imagery data. There are map
making conventions understood by cartographers in presenting
information on maps which cause maps not to be one for one mapping
of imagery. If map matching turns out to be a viable guidance
technique, then an ARPA effort in cartograahy could have considerable
inpact.
9.5 SELECTED EXAMPLES IN IMAGE UNDERSTANDING
#Major Carlstrom stated that based on the foregoing discussion, the
following specific examples in Image Understanding would serve to focus
the attention of the Workshop:
# Photo Interpretation
# Cartography
# Remotely Piloted Vehicles.
10.0 UNKNOWNS REVISITED
#Dr. Stockham questioned whether there was any confidence that in
the next ten years the sensory technology will exist to digitize
imagery to the degree required by the concepts being discussed for the
Image Understanding program. Perhaps the Image Understanding
program should try to force sensor technology.
#Dr. Licklider commented that he was very bullish on hardware
technology and he felt that the technology would be available
when needed. At the moment as many photoreceptors as are present
in the eye can be placed on a single chip which also performs some
processing. However, the Image Understanding program should stay
abreast of what is happening in hardware development and push
it if needed.
#Dr. Winston commented that Artificial Intelligence has traditionally
been all software. However, Artificial Intelligence programs have
become so complex that they take too long to run. Some hardware
preprocessing may provide a solution to this problem. He estimated
that in three years the Artificial Intelligence community might be
in a position to tell the hardware community what types of hardware
are required.
#Dr. Reddy commented that in his approach to Image Understanding
he needed the flexibility to return to the raw data in an interactive
way. Hardware implementation should consider this.
#Major Carlstrom stated that he thought that the interaction between
the signal processing and Artificial Intelligence communities was an
unknown.
#Dr. Reddy remarked that he needed help in signal processing for:
# noise removal
# preprocessing so that those things which are
important to humans are enhanced.
#Dr. Klingle commented that line by line decomposition of imagery
consumes lots of computer time and may not be the most efficient
imagery decomposition for Artificial Intelligence. The most
efficient imagery decomposition is an unknown.
11.0 APPROACH TO THE IMAGE
UNDERSTANDING PROGRAM
#Dr. Licklider stated that the current Image Processing program would
be ending in the next one to one and one half years and right now there
was a desire to incorporate Artificial Intelligence concepts into it.
Image Understanding may be a successor program to Image Processing
starting in one to one and one half years and being conducted over the
succeeding five years.
The planning for the Image Understanding program would be modelled
on the Speech Understanding program. There may be an Image
Understanding committee (perhaps this Workshop) meeting for six
months to plan the program followed by six months for contractual
procedures. However, the Speech Understanding program fell short
in the area of application planning and he did not want the Image
Understanding program to have the same short falling. Another
committee, of which he is the only current member, is beginning to talk
with Department of Defense sponsors to identify transition targets.
The planning committee will determine what can be done and the
transition committee will determine what is needed.
The needs and the accomplishable results will then be matched.
12.0 THE NEXT MEETING/
ACTION ITEMS
#Major Carlstrom stated that the next meeting would address the
approaches to the three specific examples in Image Understanding.
He desired that example imagery data in each example area be
provided to the participants at the current meeting as soon as
possible so that they could formulate their approaches and present
them at the next meeting.
It was pointed out that the Workshop also needed an appreciation
of the tasks and processes humans currently perform with the example
imagery data. Briefings at the next meeting from knowledgeable
humans should be included.
A committee was formed to make up the agenda and provide the example
imagery data for the next meeting.
The committee members are:
Dr. William Pratt
Dr. Paul Wintz
Dr. Azriel Rosenfeld
Dr. Ralph Bernstein
Dr. Martin Tenenbaum
Action items were assigned as follows:
1. The committee will collect representative photographic
interpreter data, the Wright-Patterson AFB remotely piloted
vehicle data base, and representative cartography data and make it
available both in photographic and digitized form to the
participants to the current Workshop. The digitized data will
be made available on the ARPANET.
2. The committee will have human experts familiar with the tasks
that humans currently perform with the data make presentations at the
next meeting.
3. The committee will make up the agenda for the next meeting.
The next meeting will be held in May at a place to be announced.
#Dr. Licklider encouraged those present to obtain security
clearances to make the interaction with the photographic interpreter
community easier. There is a big difference between having access
to classified background material and working on classified projects.
He suggested that those interested in obtaining security
clearances should contact Major Carlstrom.
The meeting was adjourned.
-------